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Abstract 


E-learning has become an integral part of the present day tertiary education, not only within the 
framework of combined, but, to an increasingly larger extent, also of full-time study modes. However, its 
deployment on a large scale resulted in the emergence of many problems which shall become a subject 
of research and investigation. The present study reflects the attitudes of university students toward e- 
learning within their course of study, and submits partial results of the research investigation implemented 
throughout the years 2007 to 2011. The hereinabove described survey research focused on monitoring 
and evaluating students attitudes to the teaching through e-learning based on the use of electronic study 
supports enriched with multimedia elements. The research data collection was carried out by means of a 
non-standardized research questionnaire and the data were subsequently analysed using nonparametric 


statistical methods. Selected outputs of the research, focused primarily on the assessment of the level of 


students ’ satisfaction with the organization of teaching through e-learning, and on the identification of the 
elements of electronic study materials preferred by the students, are the subject of this study. 

Key words: e-learning, e-learning support, nonparametric statistical methods, pedagogical research, 
survey research, tertiary education. 


Introduction 


As education has accompanied mankind from the earliest times, it has continually been 
developed and subject to gradual innovation. The absolute phenomenon of the late 20th and ear- 
ly 21st century has been e-learning, which consists of many sub-elements, often interlinked to 
form a comprehensive integrated system allowing for effective learning. On the one hand, these 
may be large-scale, fully distance courses using sophisticated cooperative learning tools, and on 
the other hand, they may in the nature of a mere complement to full-time study courses. 

E-learning has become an area with good prospects for growth, and as such has been 
continuously, systematically and extensively dealt with at the international level, as confirmed 
by the works stated below: 

Eom, S. B., Arbaugh, J. B. (2011), Clark, R. C., Mayer, R. E. (2011), Joo, Y. J., Lim, K. 
Y., Kim, E. K. (2011), Mauthe, A., Thomas, P. (2004), Anderson, J., McCormick, R. (2005), 
Bennett, S., Maton, K., Kervin, L. (2008), Kluge, S., Riley, L. (2008), Mohlenbrock, R. (1982), 
Paulsen, M. F. (2003), Smith, D., S., Caruso, J., B. (2010), Tapscott, D. (1998), Bates, A., Poole, 
G. (2003), Ramanau, R., Hosein, A., Jones, Ch. (2010), Zounek, J. (2009), BareSova, A. (2003), 
Nocar, D. (2004), Eger, L. (2002), Zlamalova, H. (2002), Bednarikova, I. (2008), Kopecky, K. 
(2006), Kvétoi, K. (2004), Pricha, J., Mika, J. (2000), Simonova, I. (2010). 

Since the concept of e-learning is perceived and defined in different ways, our defini- 
tion shall be unambiguous and in conformity with the context of the study. In Anglo-Saxon 
countries, the notion of e-learning (Lowenthal, Wilson, 2009) with respect to education activi- 
ties supported by ICT (e-support) is often replaced by relatively steady concepts of Computer- 
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Based Training (CBT), Internet-Based Training (IBT) or Web-Based Training (WBT) (Zounek, 
2009). In general, e-learning is understood as a type of learning, within the framework of which 
the acquisition and the use of knowledge is distributed and facilitated by electronic devices 
(Priicha, 2009). 

E-learning comprises not only a number of tools that are used for the presentation and 
the transfer of the educational content, and for the management of the studies, but also an entire 
spectrum of communication channels. The tools can be used thanks to the Learning Manage- 
ment System (LMS), which is a prerequisite for a truly effective learning process using e- 
learning. LMS therefore represents a virtual “classroom”, where learning supports, examination 
papers, learning instructions, exercise plans or discussion forums are placed (Mauth, Thomas, 
2004). Apart from LMS, among the basic tools of implementation of e-learning are properly 
structured and didactically adapted educational texts, abbreviated as e-learning supports or just 
learning supports (see Paulsen, 2003; Gray, 2010; Kopecky, 2006). Electronic study supports 
are specific in many aspects; as they are intended for a form of study, mainly characterized by 
a higher level of independence and individuality (Bates and Poole, 2003). 


Methodology of Research 


General Background of Research 


The survey research was focused on the monitoring and evaluation of students’ attitudes 
to the education realized through e-learning, based on the use of multimedia-enriched electronic 
study supports (Klement, 2010). Each electronic study support comprised a static part (verbal 
component of the text), an image part (visual component of the text) (see Mohlenbrock, 1982; 
Prucha, 1988), and a dynamic part in the nature of multimedia extension elements, animated 
solution processes, and/or interactive simulations of particular tasks (Chudy, Candik, 2004). 


Research Sample 
The research sample consisted of a total of 501 first-year students at two universities 
(Faculty of Education, Palacky University of Olomouc and Moravian College Olomouc), who 


carried out a part of their studies through e-learning. 


Table 1. Structure of the research sample. 


Year 2007 2008 2009 2010 2011 x x 




















in % 

x Women 40 39 46 56 39 220 43.9 
Bia 53 63 59 48 58 281 56.1 
> Total 93 102 105 104 97 501 

% per year 18.6 20.4 20.9 20.8 19.4 100,0 
Students satisfied with the 
organization of the educa- 95.7 92.2 94.3 94.2 94.8 472 94.2 
tion through e-learning 
Students dissatisfied with 
the organization of the edu- 4.3 7.8 5.7 5.8 5.2 29 5.8 





























cation through e-learning 
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Instrument and Procedures 


The main aim of the research was to determine the attitudes of the students toward the 
education through e-learning. Every particular part of the research was formalized into questions, 
which were then put together to create a non-standardized structured anonymous questionnaire. 
The research assumptions were subject to verification based on the use of a static nonparametric 
method, i.e. Pearson’s chi-squared test. Its aim was to determine the level of dependence of 
the outcomes of the research on the particular significant features of the particular groups of 
respondents (gender, age). To calculate the software statistical system, Statistica 9.0 was used. 

Based on the theoretical study, research investigations carried out in the field of interest 
(Schank, 2005; Anderson, McCormick, 2005; Fox, Cesal, 2008), and our own research, 
a conclusion was reached by us that distance learning implemented through e-learning is 
significantly different from the classical education. The basic presumptions are: 


e The students were satisfied with the education through e-learning because the fully 
electronic learning environment in the form of LMS met all their needs. We supposed 
that their interest was a long-term one. 

° The students preferred multimedia elements of interactive character. Their interest was 
presumed to be a consistent one. 


Results of Research 


The initial analysis was focused on the level of students’ satisfaction with the organization 
of the teaching through the use of e-learning. A research assumption was set that the students 
were satisfied with the arrangement of teaching through the use of e-learning, within the 
framework of which the educational content is provided mainly via electronic study supports 
and the communication, evaluation and managerial aspects of the study are ensured by the 
LMS system. 

After the analysis of the results obtained, it could be stated that the above mentioned 
assumption proved to be true, as 94.2% of all the respondents gave an affirmative response to 
the aforementioned question, whereas only 5.8% replied negatively. Moreover, it is possible to 
conclude that the level of the students‘ satisfaction with the organization of the teaching was a 
permanent one (see Table 1). The validity of the outcomes was subject to verification by means 
of an additional analysis, the aim of which was to determine the possible dependence of the 
result on the gender of the respondents. The chi-square test was applied for this purpose (Table 
2). 


Table 2. Level of satisfaction with the organization of education through e- 
learning - women vs. men. 





Contingency table, cell frequency > 10 marked in italics 
Pearson's chi-square > 1202, degrees of freedom: 1, significance = 0.0424 

















Gender of the respondents | am not satisfied | am satisfied Line sums 
Female 18 202 220 
Male 11 270 281 
Both groups 29 472 501 














Since the calculated level of significance is 0.04, as shown in Table 2, it can be stated 
that the frequency of the responses given by men and women with respect to the level of their 
satisfaction with the organization of the teaching through the use of e-learning are different. 
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The evaluation is partly dependent on the gender of the respondents. A possible interpretation 
of the result may be that the frequency of dissatisfied women is higher than the frequency of 
dissatisfied men. As the frequency of the dissatisfied respondents in general is rather low (the 
dissatisfied women among the students interviewed represent only 3.6% of all the respondents, 
as regards the men, a very low percentage of 2.2% was achieved), it can be stated that the 
dependence of the results on the gender was rather remarkable and called for further investigation. 
Another research assumption was therefore stated, i.e. that there was a group of students who 
clearly refused distance learning and definitely preferred full-time teaching. Another analysis, a 
comparative one this time, followed, based on the comparison of the positive and the negative 
answers given by the respondents throughout partial investigations which were aimed at the 
determination of the level of students’ satisfaction with the organization of the teaching through 
the use of e-learning. The comparative analysis consisted in analysing the above mentioned 
results and in their evaluating via the chi-square test (Table 3). 


Table 3. Level of satisfaction with e-learning as a teaching method versus the 
level of satisfaction with the organization of e-learning. 




















Contingency table, cell frequency > 10 marked in italics 
Pearson's chi-square: 155.5761, degrees of freedom: 1, significance = 0.00001 
| am not satisfied with the | am satisfied with the organi- ‘ 
eee ; . f Line sums 
organization of e-learning zation of e-learning 
| am not satisfied with e-learn- 
ing as a teaching method a a " 
| am satisfied with e-learning 8 449 457 
as a teaching method 
Both groups 29 472 501 














The calculated level of significance, as shown in Table 3, is 0.001, which indicates a high- 
level dependence of the ratings in both areas analysed. The interpretation of the result might 
be that there exists a group of students not satisfied with e-learning itself, which is identical 
with the group of students who are not satisfied with the structure of e-learning. No matter how 
small (21 students only, which is 4.2% of the total of 501 respondents), there exists a group 
of students, who reject all educational activities whatsoever carried out through e-learning. 
In order to specify this group of students, yet another, statistical analysis, aimed to determine 
the possible dependence of the results on the gender of the respondents, was carried out. The 
outcomes were verified by means of the chi-square test (Table 4). 


ISSN 1822-7864 


Milan KLEMENT, Jiri DOSTAL. E-Learning in Tertiary Education from Students’ Point of View 


Table 4. Level of satisfaction with e-learning as a teaching method versus the 
level of satisfaction with the organization of e-learning, men versus 
































women. 
Contingency table, cell frequency > 10 marked in italics 
Pearson's chi-square: 178.9271, degrees of freedom: 4, significance = 0.0001 
Women Men Line sums 
| am not satisfied with Plena : 
e-learning as a teaching | am not satisfied with the organiza- 13 8 0 
tion of e-learning 
method 
| am not satisfied with ; ; aaa 
e-learning as a teaching | am satisfied with the organization of 10 13 33 
e-learning 
method 

Total 23 21 44 

| am satisfied with e-learn- | am not satisfied with the organiza- 

: : ; F 5 3 8 

ing as a teaching method tion of e-learning 

lam satisfied with e-learn- | | am satisfied with the organization of 492 257 449 

ing as a teaching method e-learning 
Total 197 260 457 
Column totals 220 281 501 




















According to the calculated significance level, which in this case again reaches a very 
low value of 0.0001, it is possible to conclude that within the framework of the group of the 
students, who reject training through e-learning, there is a high level of dependency on the 
respondents’ gender. In fact, the ratio of women (a total of 192 students) to men (total 257 
students) who did express their satisfaction with e-learning as an appropriate form of teaching 
and with its arrangement, corresponds to the value of 0.74 within the framework of the whole 
research sample (total of 501 male students and female students). It is possible to deduce from 
this result, and from Table 4, that within the group of students who reject teaching through e- 
learning, there is a greater percentage of women than men. In fact, from the group of objectors, 
representing a total of 4.2%, the majority are women, who represent 61.5%. 

First of all, a research assumption that within the framework of the teaching with the use 
of electronic study supports, the dynamic element in the nature of an interactive educational 
animation was the favorite one among the students was subject to verification. We hereby 
reacted to the fact that the classical approach to the evaluation of electronic study supports 
(Schank, 2005), (Anderson, McCormick, 2005) does not accent all of the contemporary trends 
in the implementation of e-learning. 

The research assumption stated above proved to be true, 1.e. the dynamic element in the 
nature of an interactive educational simulation and animation did take the first place as regards 
the popularity among the students; the static element in the form of the written text came second 
only. Based on the assessment of the students’ opinions on the research issues, it can be stated 
that the development trend in this area, i.e. the views and attitudes of the students, is consistent 
and shows neither increase nor decrease (see Table 5). 
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Table 5. Students* opinions of the most suitable structural element of electronic 
study supports (percentage). 














Students’ opinions of the most suitable structural element of electronic study supports (percentage) 
2007 2008 2009 2010 2011 Average 
Static textual information (%) 36.6 30.4 37.1 30.8 39.2 34.7 
Static visual information (%) 25.8 23.5 21.0 19.2 19.6 21.8 
Dynamic visual information (%) 37.6 46.1 41.9 50.0 41.2 43.5 





























The partial results were put to yet another statistical analysis aimed at determining 
whether there existed any differences in the assessment given by men and women. As the 
verification tool, the chi-square test was used again (Table 6). 


Table 6. Students opinions of the most suitable structural element of electronic 
study supports (women versus men). 

















Contingency table, cell frequency > 10 marked in italics 
Pearson's chi-square: 0.9986, degrees of freedom: 2, significance = 0.6067 
Gender of the F Favourite elements- pic- Favourite elements F 
Favourite elements- text og Line sums 

respondents tures - animations 

Male 72 47 101 220 
Female 102 62 117 281 
Both groups 174 109 218 501 























The calculated level of significance being 0.61, it can be concluded that the frequency of 
the individual responses given by men and women with respect to their views of their favorite 
presentation element of the curriculum within the framework of the teaching process with the use 
of e-learning are identical. This particular evaluation can thus be characterized as independent 
of the gender of the respondents. 


Discussion 


Thanks to the research survey carried out, it was possible to design a tool for the 
evaluation of distance education implemented through e-learning. The emphasis was put on 
assessing the quality of electronic study materials. Even though the latter only represents one 
component of distance education through e-learning, their quality is crucial, which was also 
proved by the research survey carried out. It is clear that the quality of the LMS, applied on a 
large scale, is similar, however, it is necessary to monitor the educational content they provide 
and the effectiveness of learning they make for. Nevertheless, the other components of the 
LMS are of high importance, too, that is to say tutors or a space for discussion launched by the 
students are crucial. 

According to the modern approach applied in the evaluation of e-learning courses and/or 
study materials, “there are two very important criteria with respect to the quality of e-learning: 
it should work without problems for all participants as regards the technical aspect, and a 
straightforward application of pedagogical principles shall be evident” (Eger, 2004). Scientists 
who have treated the problem of e-learning, such as Horton (2011), only approached it in general 
terms, without paying attention to specific, practically applicable and validated assessment 
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systems, see Tham (2004), Cotton (2006), Ardito (2006). An example of a precisely elaborated 
system for the evaluation of the quality of distance education implemented through e-learning is 
the one developed by a Professor at the Department of Education of Northern Regional College, 
Newtownabbey, J. Anderson. This system aimed at the assessment of electronic study materials 
represents a high quality tool, which comprises criteria for assessing not only psychological and 
didactical attributes, but also the technical aspects of the educational process. The whole rating 
system operates with five basic categories of criteria and sixteen sub-categories (Anderson, 
McCormick, 2005). They are as follows: 


1. Provision of infrastructure: 
a) requirements for network infrastructure; 
b) requirements for equipment; 
c) specification of availability; 
d) mutual cooperation. 


2. Technical standards: 
a) functionality; 
b) design principles; 
c) the quality of activities. 


3. Content and its development: 
a) adaptation to curriculum requirements; 
b) the proposed content; 
c) planning. 


4 Pedagogical and didactical aspects: 
a) self-learning; 
b) evaluation and self-evaluation; 
c) implementation of teaching. 


5. The needs of the institution: 
a) planning and management; 
b) distribution within the institution; 
c) the accessibility and location. 


Despite its comprising highly important areas of evaluation, the system cannot be described 
as complex, since it omits some important attributes, such as psychological requirements of 
the student, distance learning patterns, ergonomic aspects of working with computers, etc. 
However, it is possible to trace efforts toward the evaluation of individual sub-elements of the 
whole educational system in the form of e-learning, whereby not only the general properties 
and overall results of the implemented course, but also individual sub-elements concerning the 
content or the structure, are taken into account. 

The practical application of the system of distance education designed and assessed 
by us consists in creating a software application that allows a fast and efficient evaluation of 
electronic study materials. The application contains a total of 42 evaluation criteria distributed 
in 6 areas in accordance with the findings of the evaluation system verification. The whole 
situation is presented in the following table number 7. 
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Table 7. The final structure of the system for the evaluation of study supports. 





Evaluated area 01: Student's personality and distance education (DIV) 

















(Criterion 1 Inducement of an emotional reaction by the student. 

Criterion 2 (Creation of real concepts with respect to the phenomena demonstrated. 
(Criterion 3 Provision of several solutions (if possible). 

Criterion 4 Provision of a sufficient number of strong impulses for the student to remember. 
Criterion 5 (Creation of real concepts with respect to the phenomena demonstrated. 





Criterion 1 


Evaluated area 02: Student's learning and distance education (DIV) 


(Classification of the contents into appropriate steps with respect to the target group of students. 





Criterion 2 Putting emphasis on the practical application of acquired knowledge. 














(Criterion 3 Possibility of verifying in practice of acquired knowledge. 

Criterion 4 Presence of learning tasks. 

(Criterion 5 Presence of the learning objectives from the emotional and psychomotor domains. 
(Criterion 6 Factual and terminological accuracy. 

Criterion 7 Adequacy with respect to the target group of students. 

(Criterion 8 Sufficient number of practical tasks. 

Criterion 9 Presence of ideas for independent work. 





Criterion 10 


Criterion 11 


or attitudes. 


Interconnection between the learning objectives and the student's final level of knowledge, skills 


Presence of learning objectives from the emotional domain. 





Criterion 12 


Presence of learning objectives from the psychomotor domain 





Evaluated area 03: Educational content and its form with respect to distance education (DIV) 























Criterion 1 Appropriate frequency of abstract concepts. 

Criterion 2 Interpretation and demonstration in a static image format (pictures). 

(Criterion 3 Interpretation and demonstrations in a dynamic image format (simulation, animation, etc.). 
(Criterion 4 (Clear and concise definition and description of the terms. 

Criterion 5 Links between the concepts explained and the previous subject matter. 

(Criterion 6 Possibility of a separate application of the concepts and the content related thereto. 
(Criterion 7 isualization of abstract and concrete concepts. 





Evaluated area 04: Particularity of distance education DiV' 




















(Criterion 1 Presence of navigation icons. 

Criterion 2 Presence of a summary of keywords 

(Criterion 3 Appropriate number of key words and their meaning with regard to the interpretation. 
Criterion 4 Objectivity and brevity of the study guide. 

(Criterion 5 Number of terms to remember, and their importance with regard to the interpretation. 
Criterion 6 Presence of the elements for an interim evaluation - short tasks (verbal response). 
(Criterion 7 Presence of the elements for a continuous self-evaluation - long tasks (file). 

(Criterion 8 Presence of the elements for a continuous self-evaluation - control questions. 





Evaluated area 05: Technical aspects of distance education (DIV) 














(Criterion 1 Fast navigation through the text (hypertext links). 

(Criterion 2 Method for the assessment of the partial results achieved (interim - final). 
Criterion 3 Possibility of an on-line testing by means of electronic tests. 

(Criterion 4 Possibility of an on-line communication via discussion groups (chat). 
(Criterion 5 Possibility of an on-line communication by means of a videoconference. 





Evaluated area 06: Ergonomic aspects of distance education (DIV) 





Presence and the adequacy of the approximate time limit designed for individual study chap- 

















(Criterion 1 

ters. 
Criterion 2 (Graphical and declarative value of the icons. 
(Criterion 3 Length and the declarative value of marginalia. 
Criterion 4 (Color layout of the text and the text hyperlinks. 
Criterion 5 Adequate length of the study support up to 60 pages. 
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Conclusion 


The outcomes stated above, which to some extent confirmed the development trends, 
required a response in the form of an analysis of the existing systems of evaluation of distance 
education implemented through e-learning, not only with respect to the overall learning 
outcomes, but also in terms of partial outcomes. The analysis of the existing rating systems 
would facilitate the assessment of the extent to which the detected and confirmed trends are 
incorporated into them, and whether or not they react to the dynamic changes influencing the 
concept of distance education, i.e. its full electronization, primarily based on electronic study 
supports incorporated in the LMS system. The former include structural elements allowing for 
the achievement of a wider range of learning objectives and they use interactivity as one of the 
basic principles of individuation and self study. 

Although being only partial, as the research investigation was carried out on an even 
larger scale, the above stated results show some trends which should modern education, using 
distance study texts in the electronic form and the LMS system, definitely take into account. 
That is why the aforementioned investigation research findings should be respected throughout 
further considerations with respect to the creation of the appropriate e-learning tools, when 
finding about the students* attitudes and views in relation to the teaching with the use of e- 
learning, and/or within the framework of the researches focused on the issue of the evaluation 
of electronic study materials. 
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